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Amendments to the Claims! 

This listing of claims will replace all prior versions, and 
listings, of claims in the application: 

Listing of Claims; 

1-51. Cancelled 

52. (Currently Amended) [A] In a video hyperlinked 
annotation data syste m, a computer-readable medium [ including a 
plurality of data Dtructuro Glcmcnta ] comprising: 

a first annotation data structure element including an 
object reference for an object in a video frame and a 
corresponding first identifier; and 

a second annotation data structure element [ rcf croncQd ] 
identified by said first identifier, said second annotation data 
structure element including a first set of annotation data 
references . 

53. (Previously Presented) The system of claim 52 further 
comprising at least one timing data indicator associated with at 
least one of said plurality of data structure elements. 

54. (Previously Presented) The system of claim 53 wherein 
said timing data indicator indicates an expiration time. 

55. (Previously Presented) The system of claim 53 wherein 
said timing data indicator' indicates an activation time. 
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56. (Currently Amended) The system of claim [i] 52_ 
wherein said first annotation data structure element is 
associated with a set of video frames of a video program and 
wherein said second annotation data structure element is 
associated with said video program, 

57. (Previously Presented) The system of claim 52 wherein 
said first data structure element and said second data structure 
element are transmitted separately. 

58. (Previously Presented) The system of claim 52 wherein 
said first set of annotation data references includes an 
annotation data field and a second identifier referencing a 
third annotation data structure element. 

59. (Previously Presented) The system according to claim 
58 wherein said annotation data field is a title data field and 
said third annotation data structure element is a string 
including a title of said object. 

60. (Previously Presented) The system according to claim 
58 wherein said third annotation data structure element includes 
at least one display identifier for referencing a fourth data 
structure element to be displayed to a viewer and at least one 
action identifier referencing a fifth data structure element 
providing instructions so said system for action to be taken by 
said system. 
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61. (Previously Presented) The system according to claim 
58 wherein said annotation data field is a variable parameter 
field. 

62. (Previously Presented) The system according to claim 
58 wherein said second identifier is a variable value. 

63. (Currently Amended) The system according to claim 
[-&3-] 58 wherein said first and second identifiers are never 
duplicated by the system. 

64. (Previously Presented) The system according to claim 
52 further comprising a fifth data structure element including 
location and shape information about said object. 

65. (Currently Amended) The system according to claim 
64[-] j_ wherein said fifth data structure element is associated 
with a video frame, 

66. (Currently Amended) A method [e#] for generating one 
or more data structures via a first processor in a hyper linked 
video signal annotation data system_^ [ including — a — plurality — e# 
data — Qtructuro — olcmcnta ] the one or more data structures being 
stored in a computer-readable medium for access by an 
application program executed by a second processor in the 
hyperlinked video signal annotation data system, the method 
comprising: 
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creating a first annotation data structure element 
including an object reference for an object in a video frame and 
a corresponding first identifier; and 

creating a second annotation data structure element 
[ rcf crcnGcd ] identified by said first identifier, said second 
annotation data structure element including a first s.et of 
annotation data references. 

67. (Previously Presented) The method of claim 66 further 
comprising creating at least one timing data indicator 
associated with at least one of said plurality of data structure 
elements . 

68. (Previously Presented) The method of claim 67 wherein 
said timing data indicator indicates an expiration time. 

69. (Previously Presented) The method of claim 67 wherein 
said timing data indicator indicates an activation time. 

70. (Previously Presented) The method of claim 66 wherein 
said first annotation data structure element is associated with 
a set of video frames of a video program and wherein said second 
annotation data structure element is associated with said video 
program. 

71. (Previously Presented) The method of claim 66 wherein 
said first data structure element and said second data structure 
element are transmitted separately. 
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72. (Previously Presented) The method of claim 66 wherein 
said first set of annotation data references includes an 
annotation data field and a second identifier referencing a 
third annotation data structure element. 

73. (Previously Presented) The method according to claim 
72 wherein said annotation data field is a title data field and 
said third annotation data structure element is a string 
including a title of said object. 

74. (Previously Presented) The method according to claim 
72 wherein said third annotation data structure element includes 

* at least one display identifier for referencing a fourth data 
structure element to be displayed to a viewer and at least one 
action identifier referencing a fifth data structure element 
providing instructions to said system for actions to be taken by 
said system. 

75. (Previously Presented) The system according to claim 
72 wherein said annotation data field is a variable parameter 
field. 

76. (Previously Presented) The method according to claim 
72 wherein said second identifier is a variable value. 
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77. (Currently Amended) The system according to claim 
[-6^] T2 wherein said first and second identifiers are never 
duplicated by the system. 

78. (Currently Amended) The system according to claim 66 
further comprisingj_ 

creating a fifth data structure element including location 
and shape information about said object. 

79. (Currently Amended) The system according to claim 
78 [-]_^ wherein said fifth data structure element is associated 
with a video frame. 

80. (NEW) The computer-readable medium of claim 52 
further comprising a third data structure storing image overlay 
data, the object being visually identified based on the stored 
image overlay data in response to a user command. 

81. (NEW) The computer-readable medium of claim 80, 
wherein the image overlay data is associated with first timing 
data, presentation of the image overlay being synchronized with 
the video frame based on the first timing data. 

82. (NEW) The computer-readable medium of claim 81, 
wherein a computer program code compares the first timing data 
with second timing data associated with a current video frame 
and sleeps for a period of time equivalent to a difference in 
times indicated by the first and second timing data, the 
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computer program code being awakened for visually identifying 
the object based on the image overlay data in response to an 
expiration of the time period. 

83. (NEW) The computer-readable medium of claim 81,. 
wherein at least one of the data structures is associated with 
second timing data, the second timing data being indicative of a 
last instance the data structure is used in a video program 
associated with the video frame, wherein the data structure is 
removed from the computer-readable medium in response to a 
determination based on the second timing data that the data 
structure is no longer used in the video program. 

84. (NEW) The computer-readable medium of claim 83, 
wherein the image overlay data and the first and second data 
structure elements are encoded into a packet stream received by 
a receiver in the video hyperlinked annotation data system. 

85. (NEW) The computer-readable medium of claim 84, 
wherein the image overlay data and the first and second data 
structure elements are encoded into a portion of the packet 
stream that is received prior to receipt, of a portion of a video 
signal including the video frame. 

86. (NEW) The computer-readable medium of claim 85, 
wherein a fixed number of packets is allocated to each of a 
plurality of image overlay data sets to be encoded into the 
packet stream, the fixed number of packets corresponding to a 
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number of packets filled by a largest image overlay data set to 
be encoded. 

87. (NEW) The computer-readable medium of claim 86, 
wherein a particular image overlay data set fills a particular 
number of packets that is less than the fixed number of packets. 

88. (NEW) The computer-readable medium of claim 87, 
wherein the first and second data structure elements are encoded 
into the packet stream by identifying a first occurrence of .the 
object associated with the first and second data structure 
elements, and moving backwards in time from the first identified 
occurrence and filling one or more unused packets allocated to 
the particular image overlay data set with at least a portion of 
the first or second data structure element. 

89. (NEW) The computer-readable medium of claim 88, 
wherein the second annotation data structure element stores an 
indication as to whether the object is associated with a video 
stream and a fourth identifier to a fourth data structure 
storing information for a particular video stream associated 
with the object, wherein a user selects the object in response 
to the object being visually identified and in response to the 
selection, a determination is made as to whether the object is 
associated with a video stream, and in response to the 
determination, an identifier of a current data structure storing 
information for a current video stream is replaced with the 
fourth identifier, wherein information for the particular video 
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stream is retrieved from the fourth data structure based on the 
fourth identifier. 

90. (NEW) The computer-readable medium of claim 88, 
wherein the packet stream is received with a video stream over a 
broadcast signal. 

91. (NEW) The computer-readable medium of claim 88, 
wherein the packet stream is received out-of-band separately 
from a video stream including the video frame. 

92. (NEW) The computer-readable medium of claim 88, 
wherein the object is visually identified via a computer usable 

* program code, the program code containing executable 
^instructions that, when executed, cause the computer to perform 
the steps of: 

identifying on a first two-dimensional section containing a 
three-dimensional volume, a first region associated with the 
object, the first region including one or more first pixels; 

filling the first region including the one or more first 
pixels with a predetermined value; 

moving in a first direction along a third dimension to a 
second two-dimensional section containing the three-dimensional 
volume; 

determining whether a second pixel in the second two- 
dimensional section that corresponds to a filled pixel in the 
first two-dimensional section belongs to the three-dimensional 
volume; 
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filling a second region including the second pixel in the 
second two-dimensional section with the predetermined value; 
returning to the first two-dimensional section; and 
moving in a second direction along the third dimension to a 
third two-dimensional section containing the three-dimensional 
volume for filling a third region. 

93. (NEW) The computer-readable medium of claim 92, 
wherein each two-dimensional section is a video frame. 

94. (NEW) The computer-readable medium of claim 92, 
wherein the third dimension is time. 

95. (NEW) A method for generating one or more data 
structures via a first processor in a video hyperlinked video 
signal annotation data system, the one or more data structures 
being stored in a computer-readable medium for access by an 
application program executed by a second processor in the 
hyperlinked video signal annotation data system, the method 
comprising : 

identifying a region of a video frame associated with an 
object included in the video frame; 

selecting a value for the identified region; 

creating a first data structure mapping the selected value 
to a plurality of pixels in the identified region, the first 
data structure further storing a first reference pointer; 
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creating a second data structure identified by the first 
reference pointer, the second data structure associating the 
value to a first identifier; and 

creating a third data structure identified by the first 
identifier, the third data structure including a first set of 
annotation data references for the object. 

96. (NEW) The method of claim 95, wherein the value is 
associated with image overlay data for visually identifying the 
object in response to a user command, 

97. (NEW) The method of claim 95, wherein the first data 
structure is associated with first timing data, the method 
further comprising processing the first timing data and 
synchronizing presentation of the image overlay with the video 
frame based on the processed first timing data. 

98. (NEW) The method of claim 97 further comprising: 
comparing the first timing data with second timing data 

associated with a current video frame; 

causing a computer program code to sleep for a period of 
time equivalent to a difference in times indicated by the first 
and second timing data; and 

awakening the computer program code for visually 
identifying the object based on the image overlay data in 
response to an expiration of the time period. 
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99. (NEW) The method of claim 98, wherein at least one of 
the data structures is associated with second timing data, the 
second timing data being indicative of a last instance the data 
structure is used in a video program associated with the video 
frame, the method further comprising: 

removing the data structure from the computer-readable 
medium in response to a determination based on the second timing 
data that the data structure is no longer used in the video 
program, 

100. (NEW) The method of claim 96, wherein the image 
overlay data and the first and second data structure elements 

. are encoded into a packet stream received by a receiver in the 
video hyperlinked annotation data system. 

101. (NEW) The method of claim 100, wherein the image 
overlay data and the first and second data structure elements 
are encoded into a portion of the packet stream that is received 
prior to receipt of a portion of a video signal including the 
video frame. 

102. (NEW) The method of claim 101, wherein a fixed number 
of packets is allocated to each of a plurality of image overlay 
data sets to be encoded into the packet stream, the fixed number 
of packets corresponding to a number of packets filled by a 
largest image overlay data set to be encoded. 
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103. (NEW) The method of claim 102, wherein a particular 
image overlay data set fills a particular number of packets that 
is less than the fixed number of packets. 

104. (NEW) The computer-readable medium of claim 103, 
wherein the first and second data structure elements are encoded 
into the packet stream by identifying a first occurrence of the 
object associated with the first and second data structure 
elements, and moving backwards in time from the first identified 
occurrence and filling one or more unused packets allocated to 
the particular image overlay data set with at least a portion of 
the first or second data structure element. 

105. (NEW) The method of claim 104, wherein the second 
annotation data structure element stores an indication as to 
whether the object is associated with a video stream and a 
fourth identifier to a fourth data structure storing information 
for a particular video stream associated with the object, 
wherein a user selects the object in response to the object 
being visually identified and in response to the selection, a 
determination is made as to whether the object is associated 
with a video stream, and in response to the determination, an 
identifier of a current data structure storing information for a 
current video stream is replaced with the fourth identifier, 
wherein information for the particular video stream is retrieved 
from the fourth data structure based on the fourth identifier. 
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106. (NEW) The method of claim 104, wherein the packet 
stream is received with a video stream over a broadcast signal, 

107. (NEW) The method of claim 104, wherein the packet 
stream is received out-of-band separately from a video stream 
including the video frame. 

108. (NEW) The method of claim 104, wherein the object is 
visually identified via a computer usable program code, the 
program code containing executable instructions that, when 
executed, cause the computer to perform the steps of: 

identifying on a first two-dimensional section containing a 
three-dimensional volume, a first region associated with the 
object, the first region including one or more first pixels; 

filling the first region including the one or more first 
pixels with a predetermined value; 

moving in a first direction along a third dimension to a 
second two-dimensional section containing the three-dimensional 
volume; 

determining whether a second pixel in the second two- 
dimensional section that corresponds to a filled pixel in the 
first two-dimensional section belongs to the three-dimensional 
volume; 

filling a second region including the second pixel in the 
second two-dimensional section with the predetermined value; 
returning to the first two-dimensional section; and 
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moving in a second direction along the third dimension to a 
third two-dimensional section containing the three-dimensional 
volume for filling a third region. 

109. (NEW) The method of claim 108, wherein each two- 
dimensional section is a video frame. 

110. (NEW) The method of claim 108, wherein the third 
dimension is time. 
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